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Four Notions of Fault for Program Specifications 


Jan A. BERGSTRA! 


Abstract 


Four notions of fault are proposed for program specifications each 
inspired by notions of fault for programs: symptomatic failure reso- 
lution fault, Laski fault, MFJ fault and regression test justification 
of change fault (RTJoC fault). Examples are provided in terms of 
the PGA style theory of instruction sequences. Each of the notions 
of fault is based on the contrast between technical specification and 
requirements specification. The latter contrast is discussed in detail. 


Keywords: technical specification, requirements specification, Laski 
fault, MFJ fault. 


1 Introduction 


The notion of a program fault has until now received far less attention from 
theorists than the somehow related notion of program correctness. The 
relation between faults and correctness is non-obvious, however, and both 
notions are quite sensitive to particularities of the respective definitions. It is 
plausible to assume that the definitions of correctness and fault guarantee 
that a correct program contains no faults. Conversely, however, there is 
no basis for the assumption that a defective (i.e. incorrect) program will 
contain one or more faults. 

Following [1, 2, 16] a fault in a program is a static property of it which 
qualifies as a cause of a failure. In the terminology of [5] an ALR fault in 
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a program X consists of (i) a failure for X, (ii) a program fragment of X 
combined with (iii) a change for said fragment, so that when the change is 
effectuated (iv) the particular failure will not occur anymore, and (v) the 
change can be understood as an improvement of the program. It is left 
unspecified when a change may be considered an improvement. 


Laski [17] made a first proposal on how to formalise a notion of im- 
provement that underlies ALR faults, thereby giving rise to a reasonably 
well-defined notion of fault, missing some quantitive information only, which 
is referred to as a Laski fault in [5]. Subsequently in [18] one finds a different 
formalisation of program faults (see also [13]), which, again following [5] is 
best understood as a second type of fault in a typology of faults. In [5] 
a third type of fault is described which is cast in terms of testing and re- 
gression testing only. The three forms of faults just mentioned differ in the 
form of justification which is given for a change, and in particular in the 
conceptualisation of a notion of the improvement brought upon by effecting 
a change. 


A change is supposed to prevent a certain failure from taking place, 
while at the same time a change must preferably not introduce new failures, 
that is failures on inputs different from the input on which a failure was 
resolved by the program being repaired. The idea for a Laski fault is that a 
corresponding change creates a correct program, the idea for an MF J fault 
is that the changed program is still working correctly on inputs where it 
was working correctly before the change. The latter idea can be relaxed 
by requiring that a given regression test suite which a program passed in 
advance of the change is still passed after the change has been made. The 
three strands of fault thus obtained merely constitute a selection from a 
larger variety of such notions. For a survey of such options I mention [5] 
and the follow-up paper [6]. 


Technical definitions for various kinds of faults depend on actual pro- 
gram notations. In [5] I made use of instruction sequences in the notation 
of the program algebra PGA of [8]. These notations are highly simplified 
and are suitable for theoretical work only. Said notions of fault each depend 
on a notion of failure which itself depends on the availability of a technical 
specification which determines which behaviours of a program are adequate 
and which behaviours fail. 
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1.1. Four Notions of Fault in a Technical Specification 


The objective of this paper is to discuss notions of fault for (technical) 
specifications of programs rather than for programs proper. In fact we will 
propose four such notions: (i) symptomatic failure resolution fault as a 
simplest notion of specification fault, (ii) Laski fault and (iii) MFJ fault 
represent different refinements of the notion of a symptomatic failure reso- 
lution fault, and (iv) regression test justification of change fault relaxes the 
constraints imposed by the notion of an MJF fault to weaker constraints 
which can be checked by means of testing only. We start with a general 
discussion of the notion of a specification fault. 


1.2 Specification Fault: General Conditions for the 
Existence of Such Faults 


For a specification to be faulty some external criterion of validity is needed. 
If a program specification constitutes the only available knowledge about 
what the program must achieve it cannot be defective other than by be- 
ing non-implementable, unreadable, too lengthy etc. The computer science 
literature provides little or no discussion of specification faults. 

I will adopt the idea that besides a technical specification for a program 
there may also be a set of requirements (requirements specification) which 
provides additional information of what is expected from a program. In the 
presence of a perceived mismatch between technical specification and re- 
quirements the suggestion that the technical specification is defective rather 
than that the requirements specification is defective can only be based on a 
conceptual asymmetry which assigns the requirements specification a higher 
status than the technical specification. 

At the same time it must be explained why the requirements specifica- 
tion would not be included as a part of the technical specification because 
when doing so a defective specification would simply turn into an inconsis- 
tent specification, which is an easy notion to imagine. We are led to the 
following thought experiment: a context C|—] is given which, when provided 
with a software component (program) P, which is yet to be designed, will 
result in an intended system CP] for which requirements specification S;eq 
is known. 

Now a specification Spec is designed with two equally important ob- 
jectives in mind: 


(i) an implementation P of Sspec will be such that C[P] satisfies S;eq; 
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(ii) Sspec is so general that its implementations can and might be used in 
other contexts just as well, and for that reason specific use cases such 
as the requirements S,¢, on C[P] are to be avoided in the specifica- 
tion Sspec- 


1.3. Technical Specifications Versus 
Requirements Specifications 


I will assume that a program P implements a fairly precise (if not formal) 
technical specification S;..,.. A technical specification tells what the pro- 
gram is supposed to achieve (compute) in a general terms, i.e. without 
any particular context or application of the program in mind. In view of 
generality a technical specification will not involve so-called use cases. If 
a program is supposed to compute a function then a logical description of 
the graph of that function may serve as a technical specification for the 
program, where it is assumed that the logical specification merely specifies 
said graph and is not biased towards any specific application or towards one 
or more specific use cases. 


We will assume that S;--7, has been designed with a specific application 
in mind. More specifically a context C|—] is known such that P will be 
used in a system of the form C|P] for which a specification is given. The 
latter specification, referred to as S;eg explains how C[P] is supposed behave 
and, w.r.t. P it has the status of a requirements specification. Typically a 
requirements specification may involve use cases, i.e. particular instances of 
desired behaviour of C[P]. A requirements specification for P may provide 
use cases for a plurality of contexts of use of P: Ci[P],...,C,[P]. 


One may prefer to consider the requirement that CP] satisfies S;eq to 
constitute a technical specification of P, therewith removing the distinction 
between specifications and requirements in this case. We will not adopt such 
a convention with the argument that C[P] sat S;eq as a property of P is not 
sufficiently application independent to qualify as a (technical) specification 
of P. Both technical specifications and requirements specifications may 
involve functional aspects as well as non-functional aspects. 


The contrast between technical specification and requirements specifi- 
cation resembles the contrast made between language constraint and appli- 
cation constraint in [14]. A similar contrast is mentioned in [15] between 
local specification and system specification. 
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1.4  Informality for Requirements Specifications 


Besides looking at a higher level of aggregation in a system, in comparison to 
a technical specification of a system component, a requirements specification 
may also sometimes use informal language. Reasons for allowing informality 
at the level of requirements may vary. Three main motives for tolerance of 
informality may be distinguished in a specific situation. Specifying these 
motives necessitates the introduction of a project supervisor who is supposed 
to be satisfied in the end with the system as developed. 


(i) Informal language is deemed more readable and more concise and for 
that reason more effective in transmitting intentions of the project 
supervisor to the engineers at work. 


(ii) By expecting readers to select most plausible interpretations the plu- 
rality of different interpretations of an informal specification may be 
limited to a lesser variety and the project supervisor may expect to 
be satisfied with (the consequences of) each of those readings. 


(iii) The project supervisor has some additional constraints in mind which 
have not yet been taken into account in the requirements specification 
and informality of the requirements introduces the flexibility to take 
said constraints into account during the development of a technical 
specification for component P. 


Below we will work with the simplifying assumption that the requirements 
specification is formal and complete. Whenever the requirements specifica- 
tion is met, an unambiguous matter in view of formality of the requirements 
specification, the project supervisor is satisfied, a matter of completeness. 


1.5 Technical Specification Faults I: The Case of 
Retrospective Specifications 


We consider the situation that a technical specification Sje-p is given for a 
given or yet to be developed program P. If P has been developed while 
subsequently the specification has been written, we speak of a retrospec- 
tive specification. The purpose of a retrospective specification is to clarify 
concisely the main properties of P. A retrospective specification may be 
used by a programmer who contemplates including P as a component in a 
system design. 
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A retrospective specification Sj¢-p, for P is defective if it is not the case 
that P sat Sion. It is hard to imagine that a problem with a retrospec- 
tive specification of a program causes a failure of that program proper, as 
without the specification being present the same dynamic behaviour would 
appear. In the case of retrospective specifications the ALR principle that 
a fault constitutes a repairable cause of a failure must be abandoned, so it 
seems. What remains is the idea that a fault constitutes a textual change 
of a specification which achieves two aspects: to do away with a certain de- 
ficiency in the specification while at the same time to lead to an improved 
specification. Unlike with program faults, however, the objective of speci- 
fication is not so much to find a correct specification (taking the assertion 
“true” for a specification suffices for that), but to determine a sufficiently 
informative correct specification. The latter notion is rather informal. We 
conclude that the idea of a Laski fault, a change of a defective artefact upon 
which a correct artefact is obtained, does not generalise in a useful manner 
to retrospective specifications. More generally we hold that when contem- 
plating notions of fault for retrospective specifications the ALR principle (a 
fault being a fragment of an artefact which can be understood as a cause 
of a failure) has to be abandoned and another intuition of fault must be 
brought to bearing. This argument motivates the following claim: 


Claim 1.1 The ALR principle that a fault constitutes a fragment of an 
artefact which can be understood as a cause of a failure cannot be used as a 
foundation for the development of notions of fault for retrospective program 
specifications. 


We intend to proceed along the lines of the ALR principle, i.e. to consider 
notions of fault which are inspired by the ALR principle. As a consequence 
we find that for retrospective specifications said endeavour is vacuous and 
our focus must be on prospective specifications instead. Returning to ret- 
rospective specifications, we propose as a second claim, that what we will 
call inverse ALR faults will play a role when contemplating faults. 


Definition 1.1 An inverse ALR fault is present in the retrospective speci- 
fication Speiro of an artefact A if 


(i) the artefact does not satisfy Sretro (t.e. not A sat Sretro); 


(it) a fragment of S;petro may be understood as a cause of the state of affairs 
mentioned under (i). 
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An inverse ALR fault blames the specification rather than the implementa- 
tion for a defect consisting of a mismatch between both. 


Claim 1.2 Notions of fault for retrospective specifications of programs can 
be based on the idea of an inverse ALR fault. 


For the idea of an inverse ALR fault the notion of causation involved is 
non-obvious and merits further investigation. 


1.6 Technical Specification Faults II: The Case of 
Prospective Specifications 


It is plausible that Sie-n has been designed with the requirement that upon 
developing an implementation P of Sj--, it will be the case that C[P] sat- 
isfies Seq in mind. In this scenario S;¢q is given in advance as well as the 
context C|—] and subsequently Sie. is developed on the basis of these data 
as a (first) step towards the development of P. Developing P in turn is 
a necessary step towards the development of C/P] which is the underlying 
engineering objective at hand. A complication which may be taken into ac- 
count is that it is plausible for S,¢, to be less formal, i.e. to a lesser extent 
sound and complete than Steen. 

Now assuming that the objective of designing S;,,, derives from the 
intention to develop P so that C[P] sat S;eq while insisting that Siecp, is 
sufficiently abstract to qualify as a technical specification, a setting emerges 
where it makes sense to think in terms of the quality of Sj... 

It will be assumed that C[—] works in such a manner that the function- 
ality of C/P] depends on the functionality of P and on no non-functional 
property of P and that if Q has the same functionality as P though with 
non-functional properties that improve those of P in some respect, the cor- 
responding non-functional properties of C[Q] may improve (in any case not 
degrade) on the corresponding non-functional properties of C[P]. 

The question then arises to what extent the notion of a fault in a spec- 
ification Si.-, makes sense, as well as the related question if specifications 
without faults can still be defective. We will propose several notions of fault 
for a technical program specification Sec, w.r.t. a requirements specifica- 
tion of the form C[—] sat 5;¢q which is required for that same program. 

The discussion will be illustrated below a very simple example pre- 
sented in terms of the PGA style theory of instruction sequences. Instruction 
sequence notations as introduced in [8] are devoid of practical significance. 
Nevertheless the example allows illustration in some detail of the various 
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notions of fault understood as theoretical concepts rather than as notions 
ready made for use in a practical setting. 


2 Specification Faults 


In this section we will discuss specification faults leading up to the notion 
of a Laski fault for specifications. 


2.1 Prospective Technical Specification as the Default for 
Specification 


Unless stated explicitly a specification for a program is assumed to be a 
technical specification rather than a requirements specification. This de- 
fault convention for the understanding of specification is in line with the 
widespread convention not to drop “requirements” from the phrase require- 
ments specification. Said default is also in line with the practice to think 
in terms of specifications of software components, where a specification is 
understood to be independent of any actual or potential use of a software 
component. Nevertheless, for the sake of clarity of exposition, I will often 
use the phrase “technical specification” and not rely on the suggested de- 
fault. In the title of the paper, however, the mentioned default is supposed 
to be taken into account. 

Moreover, a program specification is supposed to be prospective by 
default, i.e. it has an independent status which might serve or have served 
as the starting point of program development, rather than that it has come 
about by way of reverse engineering from a given software component. 


2.2 Quality Assessments for Technical Specifications 
Including Laski Faults 


Below we list some quality assessment options for a technical specification 
Stech, Serving as a specification for a program P (yet to be developed) all 
w.r.t. a given requirements specification S;¢q which expresses behavioural 
constraints (on the same program) of the form C[X] sat Sreq: 


1. A requirements specification Seq w.r.t. context C|—] is feasible if for 
some program P, C[P] sat S;eg, otherwise the requirements specifica- 
tion is infeasible. 
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2. implementable <> an implementation exists (alternatives: consis- 
tent, not self-contradictory). 


More specifically: Si¢-, is implementable if there exists a program P 
such that P sat Stech. 


Constructing a program that satisfies Sspyec is also called program 
synthesis, and (non)synthesizable may be used as an alternative for 
(non)implementable. The later terminology can be traced back at 
least to Church [12]. 


3. requirements compatible <> an implementation P of Steen exists 
such that C[P] sat Sreq. 


4. sufficient <=> implementable and moreover each implementation 
meets the requirements specification. 


More specifically: Sjecp, is sufficient if S:¢-7, is implementable and if for 
each program P such that P sat Seep, it is the case that CP] sat. S;eq. 


5. insufficient <= requirements compatible while not sufficient. 


More specifically: Si.-, is insufficient if there are programs P and 
Q such that P sat Spec and Q sat Sspec while C[P] sat Speq and 
-(C[Q| sat, Span) 


6. wrong <> implementable while not requirements compatible. 


More specifically: S;.-, is wrong if it is implementable while none of 
the implementations P of Stecn enjoys C[P] sat Speq. 


7. defective <= = not sufficient. 


8. Specification Steep is Laski k-faulty <> Ste-pn is defective and there 
exists a textual change y of S;..,, with edit distance k& or less that 
produces a specification 7(Stecn) which is sufficient w.r.t. the require- 
ments specification Seg and context C[-]. 


More specifically: given requirements specification S;¢g, a context 
C|-], and a specification Stecp, a Laski k-fault in Stecp relative to Speq 
and C|-—] is a change y (with edit length k or less), which, when ap- 
plied to Sje-n, produces an implementable specification y(Sjecn) such 
that all implementations P of 7(Stech) enjoy C[P] sat Sreq. 
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The idea of a Laski fault is simple: the specification is defective (either not 
implementable or one of its implementations fails to meet the given require- 
ments specification), and by applying a limited textual change a sufficient 
specification is obtained. We make use of the eponym Laski because it con- 
forms the approach of Laski [17] to speak of a fault of a program in case of 
a failing program in combination with a corresponding change leading to a 
correct program. We notice that the notion of a Laski fault may apply just 
as well if the specification at hand has no implementation. 

However an important distinction with the case of program faults 
arises. If a (deterministic) program contains a Laski fault it must fail on 
some input, and it will fail always on that same input, while upon the cor- 
responding change having been applied failure on the same input will not 
take place. This state of affairs warrants viewing the fault as a cause of 
the failure. 

In the case of a specification S;¢, with a Laski fault, however, the speci- 
fication may well have one or more implementations P such that C[P] sat Seq. 
Thus viewing the Laski fault in the specification Seq as a cause of failure 
is unwarranted. Rather the fault may be understood as an explanation of 
a failure of C|P] on some input s for some implementation P of S}¢-,which 
happens to feature such a failure. Stated mor formally: 


Proposition 2.1 Jf a specification Stecp, (relative to requirements Speq and 
context C'|—]) is faulty and contains a Laski k-fault y then either 


(i) the specification Stecn 1s wrong and y may be understood as a cause 
of the presence of failures in C|P] for whatever implementation P of 
Stech, or otherwise 


(ii) the specification Stecn is insufficient and y may be understood as an 
explanation of the presence of one or more failures in the behaviour 
of C|P| for one or more implementations P of Stech. 


The idea of an ALR fault as a fragment of a (program) text which causes 
a failure is replaced by what we call an E/C-ALR fault: a fragment of a 
(specification) text which for some of its implementations P explains the 
presence of a failure in the target system C[P]. The explanation indicates 
that Steen allows too much freedom of implementation thereby introducing 
the risk of certain failures in the target system. 


Definition 2.1 An E/C-ALR fault (“explain instead of cause” ALR fault) 
in an artefact is present if the artefact contains a fragment which can be 
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understood as the explanation of a failure which may arise but which need 
not arise (thereby not qualifying as a cause). 


Claim 2.1 A plurality of plausible notions of fault for prospective specifi- 
cations of programs can be obtained as detailed instances the notion of an 
E/C-ALR fault. 


The notion of a Laski fault for a technical specification seems to be the 
first and most plausible idea on the matter of specification faults. A compli- 
cation with the notion of a Laski fault is that spotting a Laski fault involves 
a mathematical proof that allows to take into account all implementations 
of the modified specification. 


2.3. Generalising Laski Faults: MFJ Faults and RTJoC Faults 


In the case of program faults two generalisations of Laski faults have been 
elaborated: an MJF fault is present if a so-called symptomatic failure can 
be removed (repaired) by applying a change which in addition is supposed 
to provide valid results whenever the original program did so. This gener- 
alisation of the notion of a Laski fault stems from [18] and has been named 
an MJF (for Mili, Frias and Jaoua) fault in [5]. 

A generalisation of the notion of an MJF fault has been outlined in [5] 
where instead of asking for an improvement of the modified program it is 
required that the modified program, besides performing better on the input 
for the symptomatic failure complies with a given regression test suite. The 
latter generalisation is referred to as a regression test justification of change 
fault (RTJoC fault) in [5). 

Below we will define counterparts for MFJ faults and for RTJoC faults 
for technical specifications. 


2.4 Proposal for the Notion of an MFJ Fault in a Technical 
Specification 


We start with an auxiliary type of fault which is then modified into the 
definition of an MFJ fault for a technical specification. 


Definition 2.2 (Symptomatic failure resolution k-fault.) Given: 


e a requirements specification Speq concerning a system C|X]| with pro- 
gram parameter X, and 
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© an implementable specification Stech for programs X, 
a symptomatic failure resolution k-fault in Stech relative to Speq, consists of 


e a change y (with edit length k or less), which is applicable to Stech 
(then obtaining y(Stecn)), and 


e an element s (symptomatic failure case) of the input domain of sys- 
tems of the form C[X], 


where the following conditions are satisfied: 
(t) Y(Stech) is implementable, 


(ii) for some program P that satisfies (implements) Stecn, C[P] fails on s 
w.r.t. Sreq; 


(iii) for no program Q that implements y(Stech), C[Q] fails on s w.r.t. Sreq- 


If a coherent specification S;,., contains a symptomatic failure resolution 
k-fault then Sjec-p is either wrong or insufficient. Moreover the following 
observation is immediate: 


Proposition 2.2 Given a symptomatic failure resolution k-fault, with symp- 
tomatic failure on s and with change y and an implementation P of Stech 
for which C|P] fails to comply with Seq on a particular input s then it 
must be the case that the computation of C|P] on s makes use of one or 
more computation(s) of P (on various inputs say t1,...,tn) which are not 
in conformance with y(Stech)- 


Thus, the upgrade of Steen to Y(Stech) May be understood as a modification 
of the specification which blocks, and thereby removes as a cause of, the 
failing computation (of C[P]) on s, and ensures that a failure on the same 
input s will not occur for any implementation of y(Stecn). 

The notion of a symptomatic failure resolution fault allows a different 
wording of the definition of Laski faults for specifications. 


Definition 2.3 (Laski k-fault.) Given the setting of Definition 2.2 an ad- 
ditional (iv)-th condition is required besides (i), (ti), and (tit): 


(iv) (sufficiency obtained) each implementation Q of y(Stech) tt is the case 
that C[P] sat Sreq- 
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Although the notion of a symptomatic failure resolution fault may seem 
convincing there is a serious problem with this notion: successive failure 
removals may undo the guarantees obtained before. This difficulty is not 
present in the case of Laski faults because upon having resolved a Laski 
fault no other faults remain by definition. In the latter observation also lies 
a weakness of the notion of a Laski fault: it is implausible to assume that an 
artefact (in this case a specification) involves a single fault only. In the case 
of programs MF J faults were introduced to take the presence of a plurality 
of faults into account. 

The idea of an MF J fault involves two aspects: (a) symptom resolution: 
a change leading to a local improvement combined with (b) preservation of 
what works well already. We propose the following generalization of MFJ 
faults for programs to specifications. 


Definition 2.4 (MF J k-fault.) Given the setting of Definition 2.2 an ad- 
ditional (iv)-th condition is required besides (i), (ti), and (iit): 


(iv) (preservation) for all inputs t for systems of the form of C|X]: if for 
each implementation Q of Stech, the computation of C|Q] on t proceeds 
in conformance with Seq, then for each implementation Q of y(Stech) 
the computation of C[Q’] proceeds in conformance with Sreq. 


A difficulty of both notions Laski fault and MFJ fault is the use of quanti- 
fiers over implementations of S'sp¢- in the respective definitions. A similar 
difficulty arises with the definition of an MFJ program fault which involves 
quantification over all inputs. 


2.5 Specification Faults Justified on the Basis of Regression 
Testing 


For practical application such quantifiers pose a problem. In [5] a testing 
based alternative for the notion of an MFJ program fault is proposed which 
avoids such quantifiers. Describing a testing based alternative for the notion 
of an MJF fault in the case of specifications is more involved and allows a 
plurality of alternatives. Our proposal for a testing based notion of fault 
reads thus: 


Definition 2.5 (Regression test justification of change specification k-fault. ) 
Given a requirements specification Speq concerning a generic program C|X], 
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with program parameter X, and an implementable but insufficient specifi- 
cation Stecn for the program parameter X, a change y (with edit length k 
or less) is an RTJoC k-fault in Stecn relative to Sreq if the conditions of a 
symptomatic failure resolution fault (see Definition 2.2) are met (with in- 
put s) and in addition the following items (iv),..,(x) with the listed properties 
are available: 


(iv) 


(v) 


(vi) 


(vii) 


(viii) 


(ix) 


(x) 


a sequence ti,...,tn of test cases for programs X (i.e. that satisfy the 
static constraints of Stech), such that for all cases the computation of 
P ont; satisfies the specification Stech, (these observations are under- 
stood as a justification of viewing P as a candidate implementation 


of Stech)s 


@ SEQUENCE $1,...,Sm (serving as a regression test suite), of test cases 
for systems of the form C[X], such that (a) the computations of C[P] 
on 8; (1 <i<m) make use of subcomputations for P exclusively with 
inputs from ty,...,tn, and (b) such that each of these computations 
(of C[P]) complies with Sreq, 


a test case s (the symptomatic fault case) for systems of the form C[X]| 
such that the computation of C[P] on s fails to comply with S;eq, while 
each of the subcomputations of P of that computation has an input in 
Atinneeyta pe 


a program P' serving as a candidate implementation of y(Stech), where 
P’ has been developed from 7(Stecn) by a team of software engineers 
who were not involved in designing P from Stech and who are unaware 
of Sreq and of the context C[-], 


a set {r1,...,%r} of test cases for programs X which extends {t1,...,tn} 
such that each of the computations of P’! on r; (1 <i <n’) complies 
with y(Stech); 


so that each of the computations of C[P’] on {s1,..., 58m, 8} makes use 
of subcomputations of P’ on inputs in {r1,...,1%n/} only, 


and so that each of the computations of C[P’] on s1,...,8m is in con- 
formance with Sreq (the regression test is passed) and in addition the 
computation of C[P’] on s is in conformance with Syeq (i.e. the failure 
of C|P| on s, serving as the symptom of fault, has been repaired). 


This regrettably elaborate definition merits various explanatory comments. 
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. It is an implicit assumption that programs P and P’ have a rea- 
sonable length. Given a specification Sj... and a set of test cases 
T = TtetstForxX it will be doable to design Pp in such a manner that it 
complies with all test cases. However, the size of Pp will be propor- 
tional to the number of test cases in T and that is implausible. Thus 
P and P’ are supposed to be “plausible” implementations of Si¢.7, and 
(Steen) which work for all relevant inputs and not just for inputs for 
P resp. P’ which occur when computing C[P] resp. C[P’] on the test 
set {51,...,Sm} and the primary symptom case s where C[P] fails. 


. It is by C[P’] passing the regression test in addition to it working in 
conformance with S;¢q that justification is claimed for referring to 
as a fault in Sje-p, with a repair into y(Stecr). 


. With the notations of Definition 2.5 we may replace Stecn by Y(Stech) 
and replace P by P’ write 841 = s, thus obtaining n + 1 test cases 
$1,---,8n41 for the generic program C[X] and take r1,..., 7p for the 
sequence as in item (now for P’) (ii), thus obtaining a setting from 
which another specification fault can be looked for and possibly be 
repaired by way of a change with k or less symbol changes. In this 
manner Definition 2.5 allows for the definition of a chain of successive 
faults with repairing changes. 


. Remarkably, defining a specification fault in the context where testing 
is the only way of obtaining information about program behaviour, 
and only such information may play a role in the definition, turns out 
to be significantly more involved than providing such a definition in 
case one is able to speak in term of programs semantics in a mathe- 
matical fashion as e.g. in Definition 2.3. 


. An advantage of Definition 2.5 over say Definition 2.4 comes from the 
observation that, assuming the availability of oracles for Sjech, ¥(Stech) 
and Sreq, all conditions and assertions that occur in the definition 
admit straightforward checks. 


. It is implicitly assumed that testing is done on the basis of test cases 
that constitute a single input (or input stream) for a program. This 
rules out metamorphic testing where two or more related inputs are 
considered and a certain relation between the corresponding outputs is 
checked. In principle Definition 2.5 can be adapted in such a manner 
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as to admit metamorphic testing as well. We will not discuss the 
details of the latter. 


3 Theory of Instruction Sequences 


I will use inSeq as an abbreviation of instruction sequence, with the con- 
notation of the instruction sequences occurring in the theory of Instruction 
sequences (also referred to as inSeq theory) as put forward in [8] and subse- 
quent work. The series of examples in the next Section is phrased in terms 
of inSeq theory. The above descriptions of notions of specification fault 
involve both existential and universal quantifiers over programs. For these 
descriptions to be watertight in al cases it ought to be stated which classes 
of programs are meant when performing such quantification. Each of the 
inSeq notations from inSeq theory can be used for that purpose, and so can 
many other program notations. 


3.1 Data and Control 


The examples below make use of so-called single bit services (also called 
Boolean services) which were introduced in full detail in [10]. A brief intro- 
duction from first principles to the use of Booelan services in inSeq theory is 
provided in [11]. Simple examples of use of single bit services can be found 
in [4]. The application of an inSeq to inputs as contained in a service family 
is given by the apply operator from [9], 

InSeq theory involves the following key elements: 


(i) PGA style program algebra, with various inSeq notations including 
PGLC, as defined in [8], and instructions for structured programming 
(conditionals and a while loop). In the example the program P is 
assumed to be written in PGLC. 


(ii) Threads, thread algebra, thread extraction, strategic interleaving. 


(iii) Services, service families; instruction sequence processing operators 
which determine the interaction between inSeq’s and services or ser- 
vice families. 


(iv) Required interfaces of inSeq’s and threads; provided interface for ser- 
vices and for service families. 
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3.2 


Disclaimer 


All developments of this paper are to be understood relative to inSeq the- 
ory, rather than to be claimed for programming in general. For instance the 
distinction made between technical specification and requirements specifica- 
tion as made above is preferably understood as being specific for the setting 
of inSeq theory. 


In this manner a seemingly unnecessary limitation of the scope of con- 


ceptual development is introduced. For doing so we have the following 
reasons /arguments: 


(a) 


(b) 


(c) 


by having the various concepts defined relative to inSeq theory con- 
tradictions with other literature (not based on inSeq theory) about 
(literally) the same notations are (trivially) avoided, 


making too broad claims, and untenable claims for that reason, is 
avoided, 


contrasts between related but different notions can be made sharper 
by providing additional emphasis or detail to some aspects of the 
definition of a concept while relaxing demands w.r.t. other aspects 
(both in comparison with the software engineering literature at large), 


if in later work a need arises to modify the definition of a notion it is 
easy to grasp how that might impact on preceding work, which then is 
limited to work that has been explicitly based on the theory of inSeq’s 
setting, 


whenever a reader considers it useful to adopt certain definitions in 
a different setting, e.g. involving other program notations and pro- 
gramming styles and methods, then importing one or more of the 
definitions while claiming adapted generality is always an option. 


An additional justification of the limitation of the scope of conceptual 
development in the paper to a fairly narrow setting lies in the idea that 
the difficulty of concept development lies not so much in providing 
the definition of a single concept but rather in developing a useful 
framework of mutually related and dependent notions. By looking 
at a framework of related notions complications and distinctions may 
arise which remain hidden otherwise. An example of such a distinction 
in the context of the paper at hand is that we will be able to introduce 
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various types of technical specification faults, while at the same time 
we will not find any basis for discussing requirements specification 
faults, and we must keep open the option that a meaningful notation 
of a requirements specification fault cannot be found. 


4 Example: Specifying a Component for a 
Program for Adding 5-bit Numbers 


We look for an inSeq X with a parameter a that acts as a part of the 
focus for various basic actions which satisfies the following requirements 
specification Sreq: 

Sreq requires first of all that for k € N, [k/a]X (the result of substitut- 
ing k for a in X) exclusively applies methods for single bit registers (Boolean 
registers) with the following foci: aux, in1:k, in2:k, out:k. In other words 
the foci for methods occurring in X are in {aux, int:a, in2:a, out:a}. 

In terms of functionality Seg requires that the inSeq Ys[X] (as given 
below) performs addition of 5 bit binary numbers a and b where a is stored 
in single bit registers (SBR’s, each with a method interface providing 16 
methods, as defined in [11]) with focus in1:1,...,in1:5 and 6 is stored in 
in2:1,...,in2:5. Outputs are to be delivered in SBR’s out:1,...,out:5. 
The initial value of register aux, meant to contain a carry bit, is arbitrary. 


Y5[X] = aux.0/0; 
ee 


(+in1:k.i/i{; [k/a]x; }{; 
+in2:k.i/i{; 
+aux.i/i{; out:k.0/0; }{; out:k.0/1; }; 
Hi 
+aux.i/O{; out:k.0/1; }{; out:k.0/0; }; 
ie 
} 
A 
This description of a generic inSeq requires some explanation. We notice 
that when X has n instructions then Ys5/X] has (n+16)-5+2 instructions. The 
carry bit placed in the single bit register under focus aux is first initialised 
to 0, then subsequently for each k € [1,5] for the pair of inputs in1:k, in2:k 
a sequence of n + 16 instructions is included which works thus: if reading 
in1:k results in value 1 then [a/k]X is performed, and otherwise 
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+aux.i/i{; out:k.0/0; }{; out:k.0/1; }; 
fat 
+aux.i/0{; out:k.0/1; }{; out:k.0/0; }; 


} 


is performed. The latter instruction sequence achieves the following: having 
checked that in1:k contains 0 the contents of the carry bit and of in2:k are 
added (in binary arithmetic) and the least significant bit is placed in out:k 
while the most significant resulting bit is placed in the SBR with focus aux. 


4.1 Further Clarification of S,., 


The description of S,eq is to some extent informal. By adding some clarifying 
remarks more precision is obtained. These remarks may be thought of as 
replies given by the project supervisor to the designer of Stech. 


1. (Initialisation of output registers.) No assumptions on initial values 
of output SBR’s are made, and that Ys[X] must work in all cases. 


2. (Details of addition w.r.t. most significant bits.) If the sum exceeds 2° 
so that 5 output bits don’t suffice it is required requiring that just 
before termination aux is set to value 1). 


3. (Specification of single bit registers.) Implicitly true is identified 
with 1 and false is identified with 0, that may be made explicit. 


4. (Choice of inSeq notation.) A plausible inSeq notation is PGLC (ter- 
mination does not require ! and takes) place once the position after 
the last instruction is reached). Moreover it is plausible to state ex- 
plicitly that @ can occur as a parameter only in the following foci: 
in1:a, in:2a, aux:a, and in no other manner, 


With these clarifications we hold that the relation Ys[X] sat Syeg is rigorously 
defined as a property of X. 
4.2 S°.., a Plausible Technical Specification 


Carrying on with the example a plausible technical specification S?.., of X 
is thus: 
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Stech a Stnterface® Stunctional with 
Sceiaee = (Tveqiiveal®) = aux. Inethoa(SBR) 
U{ini:a.i/i, in2:a.i/i, out:a.0/0, out:a.0/1}, and 
So nctional = (post(out:a) = (pre(in2:a) + pre(aux) + 1) mod 2 A 
((pre(aux)) = 1 A pre(in2:a) = 1) > post(aux) = 1) A 
((pre(aux)) = 1 A pre(in2:a) = 0) > post(aux) = 1) A 
((pre(aux)) = 0 A pre(in2:a) = 1) > post(aux) = 1) A 
((pre(aux)) = 0 A pre(in2:a) = 0) > post(aux) = 0) A 
post(ini:a) = pre(inl:a) \ post(in2:a) = pre(in2:a)). 
Here pre(f) equals the content of the service (SBR) in focus f just before 
performing X and post(f) is the content of the service (SBR) in focus f just 
after performing X. 
Proposition 4.1 S°.., is implementable and allows the following imple- 
mentation in the inSeq notation PGLC extended with instructions for the 
conditional construct, as outlined in [8]: 
P= 
+in2:a.i/if{; 
+aux.i/1{; out:a.0/1; }{; out:a.0/0; }; 
HG 
+aux.i/i{; out:a.0/0; }{; out:a.0/1; }; 


} 


Proposition 4.2 With P as in Proposition 4.1 it is the case that Ys5[P] sat Sreq. 


Proposition 4.3 Sspec is sufficient w.r.t. Sreq and Ys[—]. 


4.3. Weakening S?.., (e.g. to Si,., Below) May Introduce a 
Laski Fault 


In St.., below it is possible that the carry is set to 1 in case it ought to be 
set to (or kept at) 0. 
Stech a Sinbeveanue Seuastitouan with 
Stinctional = (post(out:a) = (pre(in2:~) + pre(aux) + 1) mod 2 A 
((pre(aux)) = 1A pre(in2:a) = 1) > post(aux) = 1) A 
((pre(aux)) = 1A pre(in2:a) = 0) > post(aux) = 1) A 
((pre(aux)) = 0 A pre(in2:a) = 1) > post(aux) = 1) A 
((pre(aux)) = OApre(in2:a) = 0) > post(aux) = post(aux)) A 
post(ini:a) = pre(inil:a) A post(in2:a) = pre(in2:a)). 


aux 
au. 
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It is easy to imagine an implementation Q of S{,,, such that 4(Y5[Q] sat Sreq), 
for instance if the carry aux is always set to 1 by Q. It follows that 


Proposition 4.4 (i) St.4, is implementable. (Any implementation of 
So .cn also implements Sten); 


(ii) Stecn is not wrong (same argument), 


(itt) Steg, is insufficient. (One may imagine an implementation P of 
Soch where the carry is always set to 1) in that case the addition 
of (1,0,0,0,0) and (0,0,0,0,0) produces: (1,1,0,0,0,) which consti- 
tutes a failure for Ys[P]), 


(iv) Sie, has a9—Laski fault. (Indeed by changing post(aux) = post(aux)) 
to post(aux) = 0 one may transform Sounctionar MtO Stunctionar @ SUf- 
ficient specification is obtained.) 


(v) Siocn has a 2—Laski fault (assuming that multiplication is allowed in 
the specification notation). (Indeed by changing post(aux) = post(aux)) 
to post(aux) = 0- post(aux) one may transform Sounctionay into a 
specification which logically equivalent to Stunctionar and therefore is 
a sufficient specification. It also follows that the presence and size of 
specification faults depends on the specification notation.) 


4.4 Modifying S°,., to Si2,, Renders it Wrong 


In S{2.,, below the carry is set to 1 in case it ought to be set to (or kept at) 
0. 


1b — ad 1b : 
Stech — Sintereave Sfunctional with 


Sfoneeionad = (post(out:a) = (pre(in2: 
((pre(aux)) = 1A pre(in2 


( ( — post(aux) = 1) A 
((pre(aux)) = 1/ pre(in2 
, ) 


) 

) > post(aux) = 1) A 
((pre(aux)) = 0 A pre(in2 ) > post(aux) = 1) A 
((pre(aux)) = 0A pre(in2 LSTA 


post(int:a) = pre(inl:a) A post 


— post(aux ) 
in2:a) = pre(in2:aq)). 


Proposition 4.5 (i) Sty, is implementable, 
(ii) St®., is wrong (adding (1,0,0,0,0) and (0,0,0,0,0) will fail), 


(iti) S$2, has a 1—Laski fault. (Indeed by changing a single 1 to 0 the 
sufficient specification S°,., is regained.) 
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4.5 Another Wrong Specification 


In Sié_.,, the arithmetic of addition is mistaken (+1 is missing) with the con- 
sequence that each implementation of S{S., fails to satisfy (Ys[Q] sat Sreq). 
sic, =S 1 with 


he ean BS aeiant 


(post(out:a) = (pre(in2:a 

((pre(aux)) = 1 V pre(in2:a) = 1) y=) 
((pre(aux)) = 0 A pre(in2:a) = 0) > post(aux) = 1) 
((pre(aux)) = 0 A pre(in2:a) = 1) > post(aux) = 1) 
((pre(aux)) = 0 A pre(in2:a) = 0) + post(aux) = 0) 
post(ini:a) = pre(inl:a) \ post(in2:a) = pre(in2:a)). 


Sruchional = = 


+ pre(aux)) mod 2 A 
= 1) > post(aux) = 


A 
A 
A 
A 


Proposition 4.6 SiS... involves a 2—Laski fault. (By adding a summand +1 
an implementable and sufficient specification (i.e. S°)tecn) is obtained). 


4.6 Further Weakening S},., May Introduce One or More 
MFJ Faults 


In $2,., below it both possible that the carry is set to 1 in case it ought to 
be set to (or kept at) 0 and the other way around. 


Steen = =, Sentevtucutt Sfictaanat with 
S2 nctional = (post(out:a) = (pre(in2:a) + pre(aux) + 1) mod 2 A 
((pre(aux)) = 1A pre(in2:a) = 1) > post(aux) = 1) A 


((pre(aux)) = OApre(in2:a) = 1) — post(aux) = post(aux)) A 
((pre(aux)) = OApre(in2:a) = 0) > post(aux) = post(aux)) A 
post(ini:a) = pre(ini:a) A post(in2:a) = pre(in2:a)). 


( )) 
((pre(aux)) = 1 A pre(in2:a) = 0) > post(aux) = 1) A 
fated) 


Proposition 4.7 (i) $2.4, is implementable. (Any implementation of 
Si.cn also implements $2.4), 


(ii) Seog, is not wrong (same argument), 
(iti) Seog, is insufficient. (Immediate because Strecn is insufficient), 


(iv) Sec, has a 1-MFJ fault. (Indeed by changing a single 1 to 0 the 
specification Si, can be regained. ), 


Proof: Only (iv) requires attention. Consider an implementation P of 
the following specification 


2b 2b q 
Stech = ~ Sr ecrice™ Prunctional with 
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+ pre(aux) +1) mod 2 A 
1) + post(aux) = 1) A 
0) > post(aux) = 1) A 
1) + post(aux) = 0) A 
) J=1yA 
post(ini:a) = pre(inl:a) \ post(in2:a) = pre(in2:a)). 
Now P is an implementation of $2,., as well because $28... refines S2,cn- 
Consider input s = (a,b) with a = (1,0,0,0,0) and b = (1,0,0,0,0) for 
Y5[P]. On this input pair s the result of the computation of Ys[P] will be 
(0,0,0,0,0) which is not in conformance with Syeg. We find that s is a 
symptomatic failure case for P. Moreover by changing (with an appropri- 
ate y with edit distance 1) post(aux) = 0 to post(aux) = 1 that par- 
ticular failure is resolved, though the resulting specification specification 
NS anee) = Se say with implementation Q, is still wrong, according to 
Proposition 4.5. Now 7(S2,¢-) has a 1-Laski fault which can be repaired by 
changing the last instance of post(aux) = 1 into post(aux) = 0 thereby 
regaining $°.cy- 


SP acesona1 = (post(out:a) = (pre(in2:a) 
pre(aux)) = 1 A pre(in2:a) 
:Q) 
:Q) 


= ( 
(( ( 
((pre(aux)) = 1/ pre(in2 
((pre(aux)) = 0 A pre(in2 
( 


(pre(aux)) = 0 A pre(in2:a) = 0) > post(aux 


4.7 A Connection With Regression Testing 


The specification $?,., and its implementation P with change 7(SZ,,,) and 
symptomatic failure s and a regression test suite consisting of a single test s’ 
for adding a’ = (0,0,0,0,0) and b = (0,0,0,0,0) with result (0,0,0,0,0) 


provides an example of a 1—-RTJoC fault for $2... 


5 Concluding Remarks 


For the notion of a program specification we may make use of classical lit- 
erature such as [3, 15, 14]. Although less common such specifications may 
involve performance characteristics as well. In more than mere functional 
properties are specified from a program we may speak of an extended func- 
tional (EF) specification. 

Some authors, however, claim that a specification determines for a 
software component X “how it works”, and might prefer to refer to an 
EF-specification as a requirements specification instead. Understanding of 
specification as being about the how rather than the what is mentioned as 
the second option in [19], and is also mentioned as an existing viewpoint 
in [15]. If, however, one understands a specification as a description of how 
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a program works, the notion of a failure of compliance with the specifica- 
tion acquires a different meaning and the notion of a fault becomes quite 
detached from observable program behaviour. 


Therefore in the paper it is assumed that a specification P is in fact 
a functional specification or more generally an EF-specification. Having 
looked into the details of program faults, cast in the setting of instruction 
sequences, it seems plausible to contemplate the notion of a specification 
fault. As it turns out, however, unlike with the case of program faults we 
were unable to find any existing work on specification faults, apart from 
the idea that if a specification has no implementation something must be 
wrong. In [7] the notion of an algorithm fault is defined, a definition which 
is indirectly based on the notion of a program fault. 


For a specification to be considered faulty some reference to another 
description of constraints for a program is needed, and in fact the other 
description must be attributed a higher status. We have chosen to speak 
of technical specifications and to introduce requirements specifications as 
the label for the background against which a claim that a specification is 
faulty is to be justified. Moreover we have chosen to assign requirements 
specifications the role of use case descriptions which for that reason are too 
specific for a certain application to be included as a part of a program spec- 
ification. Under these assumptions we find that the theoretical framework 
for defining and analysing program faults as pioneered by [17] and [18] can 
be generalised to the case of program specifications. 


An obvious question, which we have not attempted to solve is to find 
out whether in the existing body of practical software specifications the 
different notions of fault as identified in the paper can be recognised. An- 
other topic for future work is to investigate to what extent notions of fault 
for technical specifications can be maintained once requirements specifica- 
tions are allowed to be informal rather than formal. A further question 
might be to investigate to what extent is makes sense to speak of faults 
in a requirements specification. The current paper gives no clue on how 
to conceptualise faults in a requirements specification. Intuitively however, 
one may easily imagine a failure taking place during the phase of require- 
ments capture which then becomes inadvertedly codified in a requirements 
specification document. 
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